Learning-based Automatic Parameter Tuning for Big Data Analytics Frameworks

Big data analytics frameworks (BDAFs) have been widely used for data processing applications. These frameworks provide a large number of configuration parameters to users, which leads to a tuning issue that overwhelms users. To address this issue, many automatic tuning approaches have been proposed. However, it remains a critical challenge to generate enough samples in a high-dimensional parameter space within a time constraint. In this paper, we present AutoTune--an automatic parameter tuning system that aims to optimize application execution time on BDAFs. AutoTune first constructs a smaller-scale testbed from the production system so that it can generate more samples, and thus train a better prediction model, under a given time constraint. Furthermore, the AutoTune algorithm produces a set of samples that can provide a wide coverage over the high-dimensional parameter space, and searches for more promising configurations using the trained prediction model. AutoTune is implemented and evaluated using the Spark framework and HiBench benchmark deployed on a public cloud. Extensive experimental results illustrate that AutoTune improves on default configurations by 63.70% on average, and on the five state-of-the-art tuning algorithms by 6%-23%.Comment: 12 pages, submitted to IEEE BigData 201

Local block multilayer sparse extreme learning machine for effective feature extraction and classification of hyperspectral images.

Although extreme learning machines (ELM) have been successfully applied for the classification of hyperspectral images (HSIs), they still suffer from three main drawbacks. These include: 1) ineffective feature extraction (FE) in HSIs due to a single hidden layer neuron network used; 2) ill-posed problems caused by the random input weights and biases; and 3) lack of spatial information for HSIs classification. To tackle the first problem, we construct a multilayer ELM for effective FE from HSIs. The sparse representation is adopted with the multilayer ELM to tackle the ill-posed problem of ELM, which can be solved by the alternative direction method of multipliers. This has resulted in the proposed multilayer sparse ELM (MSELM) model. Considering that the neighboring pixels are more likely from the same class, a local block extension is introduced for MSELM to extract the local spatial information, leading to the local block MSELM (LBMSELM). The loopy belief propagation is also applied to the proposed MSELM and LBMSELM approaches to further utilize the rich spectral and spatial information for improving the classification. Experimental results show that the proposed methods have outperformed the ELM and other state-of-the-art approaches

Object-based attention mechanism for color calibration of UAV remote sensing images in precision agriculture.

Color calibration is a critical step for unmanned aerial vehicle (UAV) remote sensing, especially in precision agriculture, which relies mainly on correlating color changes to specific quality attributes, e.g. plant health, disease, and pest stresses. In UAV remote sensing, the exemplar-based color transfer is popularly used for color calibration, where the automatic search for the semantic correspondences is the key to ensuring the color transfer accuracy. However, the existing attention mechanisms encounter difficulties in building the precise semantic correspondences between the reference image and the target one, in which the normalized cross correlation is often computed for feature reassembling. As a result, the color transfer accuracy is inevitably decreased by the disturbance from the semantically unrelated pixels, leading to semantic mismatch due to the absence of semantic correspondences. In this article, we proposed an unsupervised object-based attention mechanism (OBAM) to suppress the disturbance of the semantically unrelated pixels, along with a further introduced weight-adjusted Adaptive Instance Normalization (AdaIN) (WAA) method to tackle the challenges caused by the absence of semantic correspondences. By embedding the proposed modules into a photorealistic style transfer method with progressive stylization, the color transfer accuracy can be improved while better preserving the structural details. We evaluated our approach on the UAV data of different crop types including rice, beans, and cotton. Extensive experiments demonstrate that our proposed method outperforms several state-of-the-art methods. As our approach requires no annotated labels, it can be easily embedded into the off-the-shelf color transfer approaches. Relevant codes and configurations will be available at https://github.com/huanghsheng/object-based-attention-mechanis

LibraryGuru: API recommendation for Android developers

Ministry of Education, Singapore under its Academic Research Funding Tier

On the topological surface states of the intrinsic magnetic topological insulator Mn-Bi-Te family

We review recent progress in the electronic structure study of intrinsic magnetic topological insulators (MnBi$_2$Te$_4$)(Bi$_2$Te$_3$)$_n$ ($n=0,1,2,3$) family. Specifically, we focus on the ubiquitously (nearly) gapless behavior of the topological surface state Dirac cone observed by photoemission spectroscopy, even though a large Dirac gap is expected because of surface ferromagnetic order. The dichotomy between experiment and theory concerning this gap behavior is perhaps the most critical and puzzling question in this frontier. We discuss various proposals accounting for the lack of magnetic effect on the topological surface state Dirac cone, which are mainly categorized into two pictures, magnetic reconfiguration, and topological surface state redistribution. Band engineering towards opening a magnetic gap of topological surface states provides great opportunities to realize quantized topological transport and axion electrodynamics at higher temperatures

API recommendation for event-driven Android application development

Ministry of Education, Singapore under its Academic Research Funding Tier

Dimensionality reduction based on determinantal point process and singular spectrum analysis for hyperspectral images

Dimensionality reduction is of high importance in hyperspectral data processing, which can effectively reduce the data redundancy and computation time for improved classification accuracy. Band selection and feature extraction methods are two widely used dimensionality reduction techniques. By integrating the advantages of the band selection and feature extraction, the authors propose a new method for reducing the dimension of hyperspectral image data. First, a new and fast band selection algorithm is proposed for hyperspectral images based on an improved determinantal point process (DPP). To reduce the amount of calculation, the dual-DPP is used for fast sampling representative pixels, followed by k-nearest neighbour-based local processing to explore more spatial information. These representative pixel points are used to construct multiple adjacency matrices to describe the correlation between bands based on mutual information. To further improve the classification accuracy, two-dimensional singular spectrum analysis is used for feature extraction from the selected bands. Experiments show that the proposed method can select a low-redundancy and representative band subset, where both data dimension and computation time can be reduced. Furthermore, it also shows that the proposed dimensionality reduction algorithm outperforms a number of state-of-the-art methods in terms of classification accuracy

MIMN-DPP: Maximum-information and minimum-noise determinantal point processes for unsupervised hyperspectral band selection

Band selection plays an important role in hyperspectral imaging for reducing the data and improving the efficiency of data acquisition and analysis whilst significantly lowering the cost of the imaging system. Without the category labels, it is challenging to select an effective and low-redundancy band subset. In this paper, a new unsupervised band selection algorithm is proposed based on a new band search criterion and an improved Determinantal Point Processes (DPP). First, to preserve the original information of hyperspectral image, a novel band search criterion is designed for searching the bands with high information entropy and low noise. Unfortunately, finding the optimal solution based on the search criteria to select a low-redundancy band subset is a NP-hard problem. To solve this problem, we consider the correlation of bands from both original hyperspectral image and its spatial information to construct a double-graph model to describe the relationship between spectral bands. Besides, an improved DPP algorithm is proposed for the approximate search of a low-redundancy band subset from the double-graph model. Experiment results on several well-known datasets show that the proposed optical band selection algorithm achieves better performance than many other state-of-the-art methods

Medical Imaging Education in Biomedical Engineering Curriculum: Courseware Development and Application through a Hybrid Teaching Model*

4 Halama